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(54) Red-eye detection method 

(57) A red-eye detection method includes receiving 
digital image data that defines an image. The digital im- 
age data are processed to identify all regions of the im- 
age that include a specular reflection. Image regions 
deemed to include a specular reflection are processed 



further, according to conventional techniques, to deter- 
mine the presence or absence of red-eye. Specular re- 
flections are identified according to luminace-chromi- 
nance characteristics, geometric (e.g., size/shape) 
characteristics, and/or luminance gradient characteris- 
tics. 
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Description 

Background of the Invention 

[0001] The present invention relates to a novel and 5 
non-obvious image processing method for detecting po- 
tential red-eye in an image. The invention is not directed 
to a method for correcting red-eye once detected, as nu- 
merous red-eye correction methods are well known in 
the art. The present invention provides a faster and w 
more accurate red-eye detection method or potential 
red-eye detection method without operator intervention, 
and is adapted for use upstream relative to a conven- 
tional red-eye detection/correction system. 
[0002] Red-eye is a common phenomenon apparent 15 
in photographic images (whether continuous tone or 
digital) taken using a flash or strobe, wherein a human 
subject's eyes appear "blood red" instead of their natural 
color. Red-eye is caused by a reflection of the flash light 
from the retina of the eye back into the objective lens of 20 
the camera. 

[0003] Red-eye has become more prevalent and se- 
vere as cameras have been made smaller with integrat- 
ed flashes. The small size coupled with the built-in na- 
ture of the flash requires placement of the flash in close 25 
proximity the objective lens. Thus, a greater portion of 
the reflected light from a subject's retinas enters the ob- 
ject lens and is recorded. Recognizing this problem, 
camera manufacturers have attempted to minimize or 
inhibit red-eye by equipping cameras with the ability to 30 
emit one or more pre-flashes of light immediately prior 
to completion of the actual photograph. These pre-flash- 
es are intended to constrict the subject's pupils to min- 
imize light incident on the retina and reflected therefrom. 
While these modern cameras do reduce the occurrence 35 
of red-eye, it continues to be a common and bothersome 
occurrence. 

[0004] As noted above, image processing techniques 
are known and utilized to detect and correct red-eye. 
Prior methods are disclosed, for example, in the follow- *o 
ing U.S. Patents, the disclosures of which are hereby 
expressly incorporated by reference: 5,990,973; 
5,130,789; 6,016,354; 5,153,632; 5,202,719; 
5,404,192; 5,432,863; - 5,666,215; 5,698,379; 
5,747,228; 5,748,764; 5,804,356; and, 5,840,470. 45 
[0005] In general, conventional image processing 
techniques for detecting or identifying red-eye have re- 
quired human intervention or have been highly ineffi- 
cient in terms of required image processing operations. 
In one prior method, an operator must visually scan ail 50 
images and mark those images including red-eye forfur- 
ther processing. This is time-consuming and expensive. 
Attempts to eliminate or reduce operator involvement 
have resulted in automated processes that attempt to 
detect red-eye based upon color, size, and shape crite- 55 
ria. Given that the color red is very common, and given 
that red-eye is not present in a great many images (e. 
g., those not taken using a flash, those not including hu- 



man subjects, etc.) vast amounts of image processing 
resources are used inefficiently for this purpose. Fur- 
thermore, false-positives are common owning to the fact 
that every red object satisfying the size/shape criteria 
will be identified as red-eye. Thus, red buttons, a piece 
of red candy, etc., all may be misidentified as red-eye 
using these prior automated red-eye detection tech- 
niques. 

[0006] In light of the foregoing specifically noted defi- 
ciencies and others associated with conventional red- 
eye detection techniques, it has been deemed desirable 
to develop and novel and non-obvious red-eye detection 
technique that overcomes these deficiencies while pro- 
viding better overall results. 

Summary of the Invention 

[0007] In accordance with the present invention, a 
red-eye detection method includes receiving digital im- 
age data that defines an image. The digital image data 
are processed to identify all regions of the image that 
include a specular reflection. Image regions deemed to 
include a specular reflection are processed further, ac- 
cording to conventional techniques, to determine the 
presence or absence of red-eye. Specular reflections 
are identified according to luminace-chrominance char- 
acteristics, geometric (e.g., size/shape) characteristics, 
and/or luminance gradient characteristics. 
[0008] One advantage of the present invention is the 
provision of an automated red-eye detection method 
that requires less image processing resources relative 
to conventional automated red-eye detection methods. 
[0009] Another advantage of the present invention re- 
sides in the provision of a red-eye detection method that 
first identifies image regions that potentially include red- 
eye incorporating less resource intensive methods, 
wherein intensive image processing for red-eye detec- 
tion is performed on these areas only. 
[001 0] A further advantage of the present invention is 
found in the provision of a red-eye detection method 
wherein color is not or not dominantly used as an indi- 
cation of an image region potentially including red-eye. 
[0011] Still another advantage of the present inven- 
tion is found in the provision of a red-eye detection meth- 
od that identifies potential red-eye regions of an image 
upstream from a conventional red-eye detection and/or 
correction system. 

[0012] According to a further embodiment a method 
as defined in claim 10 is provided. 
[0013] In a further embodiment the step of identifying 
all regions of said digital image including a specular re- 
flection comprises: identifying all regions of said image 
including a potential specular reflection; determining a 
geometric attribute of said potential specular reflection; 
comparing said geometric attribute to a desired geomet- 
ric attribute; and, identifying said region as a region in- 
cluding a specular reflection if said geometric attribute 
conforms to said desired geometric attribute. 
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[0014] In a further embodiment the step of determin- 
ing a geometric attribute comprises determining at least 
one of a size attribute and a shape attribute, wherein 
said comparing step comprises comparing said geomet- 
ric attribute to a desired size attribute and shape at- 
tribute, respectively. 

[0015] In a further embodiment the method further 
comprises: determining a luminance and a chrominance 
of said potential specular reflection; comparing said lu- 
minance to a luminance threshold; comparing said 
chrominance to a chrominance range; identifying said 
region as a region including a specular reflection if said 
geometric attribute conforms to said desired geometric 
attribute, said luminance is greater than said luminance 
threshold, and said chrominance is within said chromi- 
nance range. 

[0016] In a further embodiment the method further 
comprisesrdetermining a luminance gradient of said po- 
tential specular reflection; comparing said luminance 
gradient to a luminance gradient threshold; and, identi- 
fying said region as a region including a specular reflec- 
tion if said geometric attribute conforms to said desired 
geometric attribute, said luminance is greater than said 
luminance threshold, said chrominance is within said 
chrominance range, and said luminance gradient is 
above said luminance gradient threshold. 
[001 7] In a further embodiment the step of determin- 
ing a luminance gradient comprises a subtraction oper- 
ation to determine a rate of change of luminance for said 
potential specular reflection. 

[0018] Still other benefits and advantages of the 
present invention will become apparent to those of or- 
dinary skill in the art to which the invention pertains upon 
reading the following specification. 

Brief Description of the Drawings 

[0019] The invention comprises a plurality of steps 
and arrangements of steps, a preferred embodiment of 
which is disclosed herein with reference to the accom- 
panying drawings that form a part hereof and wherein: 

FIGURE 1 is a diagrammatic illustration of a human 
eye exhibiting the red-eye phenomenon; 
FIGURE 2 is a partial diagrammatic illustration of a 
image including the eye of FIGURE 1 and other ob- 
jects; 

FIGURE 3 is a high-level flow chart that discloses 
a red-eye detection method in accordance with the 
present invention; and, 

FIGURE 4 is a more detailed flow chart illustrating 
sub-steps of the red-eye detection method dis- 
closed in FIGURE 3. 

Detailed Description of the Preferred Embodiment 

[0020] Referring now to the drawings, wherein the 
showings are for the purpose of illustrating a preferred 



embodiment of the invention only and not for limiting 
same, FIGURE 1 diagrammatically illustrates a human 
eye E including an iris I, and a pupil P, both of which are 
covered by a transparent cornea. For the purpose of this 
5 description, we will assume that the image of the eye 
contains said red-eye effect. This effect is caused by the 
reflection of the flash light from the retina at the back of 
the eye. Considering the shape of a human eye, one 
can deduce that a reflection from the front of the eye 
should also be visible in close spatial proximity to the 
reflection from the back side of the eye. The front side 
reflection is a reflection from a smooth, moist surface, 
thereby exhibiting ail characteristics of a specular reflec- 
tion. The illustrated eye E also exhibits a specular re- 
flection SR, in the example overlapping with the spatial 
extent of the pupil P. The specular reflection is a brilliant, 
essentially "white" area that results from the reflection 
of a photographic flash or strobe from the cornea of the 
eye. As is well known, red-eye is evidenced by a blood 
red color of the pupil P in an image. 
[0021] FIGURE 2 illustrates a partial image A defined 
by a background B, a first object 01, a second object 
02, a third object 03, and the eye E described above. 
The image is preferably provided in digital form or, if not, 
is converted from continuous tone data to digital data 
for processing in accordance with the present invention. 
Most preferably, the image data is digital image data de- 
fined by a plurality of pixel values expressed in terms of 
a luminance-chrominance color space or converted into 
a luminance-chrominance color space such as YCrCb, 
CIELABorthelike. 

[0022] Referring now also to FIGURE 3, the red-eye 
detection method in accordance with the present inven- 
tion is disclosed. The method is implemented on any 
suitable image processing or photofinishing apparatus 
and includes a step S1 of receiving digital image data 
that defines an image, e.g., the image A. A step S2 in- 
cludes processing the digital image data to identify all 
regions R thereof that include a specular reflection SR. 
Finally, in step S3, only those regions R that are deemed 
to include a specular reflection SR are processed fur- 
ther, according to conventional techniques, to determine 
the presence or absence of red-eye. Thus, with refer- 
ence to FIGURE 2, the step S2 would result in a region 
R1 (shown in broken lines for clarity) being identified as 
the only region R of the image A including a specular 
reflection SR. 

[0023] Those of ordinary skill in the art will recognize 
that the present invention capitalizes on the fact that es- 
sentially all red-eye is associated with a specular reflec- 
tion SR, i.e., without a specular reflection, there will be 
no red-eye. On the other hand, the presence of a spec- 
ular reflection does not necessarily indicate the pres- 
ence of red-eye. In fact, many times a specular reflection 
SR will be present without red-eye. Thus, further con- 
ventional processing of the regions R identified as in- 
cluding a specular reflection SR is required to determine 
the presence or absence of red-eye associated with the 
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specular reflection SR in the region R. Conventional 
processing to detect and correct red-eye within each re- 
gion R including a specular reflection SR is carried out 
by any suitable method, such as those set forth in the 
above-identified prior patents. Often, these methods 5 
identify red-eye based upon locating an image feature 
or object having a particular red color with a particular 
size and shape corresponding to the size and shape of 
a human pupil P. 

[0024] The step S2 of identifying all regions R includ- 
ing a specular reflection SR includes a plurality of sub- 
steps S2a-S2f in accordance with the present invention 
as disclosed in FIGURE 4. Those of ordinary skill in the 
art will recognize that some of the steps set forth in FIG- 
URE 4 can be carried out in an order different from that 
specified herein without departing from the overall 
scope and intent of the present invention. 
[0025] The sub-step S2a includes processing the da- 
ta or pixels defining the image A and identifying all re- 
gions R including potential specular reflections PSR 
based upon luminance-chrominance data, i.e., all bright 
white regions are identified as potential specular reflec- 
tions PSR. In one example, using the YCrCb 8bit/sepa- 
ration color space description, such bright white regions 
can be identified by locating a group of adjacent pixels 
satisfying all of the following relationships: Y > 220; -16 
< Cb < 16; and, -8 < Cr < 24. For example, in FIGURE 
2, the specular reflection SR would be identified as a 
potential specular reflection. Also, for example, the ob- 
jects 01-03 would also be identified as a potential spec- 
ular reflection based upon their bright white luminance- 
chrominance characteristics. 

[0026] A sub-step S2b then compares each potential 
specular reflection PSR to a geometric attribute, for ex- 
ample a desired shape and/or size. For simplicity of 
processing in the description, the PSR is compared to 
a selected size threshold , e.g. pixel height and width val- 
ues above which the potential specular reflection is 
deemed too large to be a specular reflection. Thus, with 
reference to the image A in FIGURE 2, the object 01 
and 02 exceed the size threshold and are discarded 
from the set of potential specular reflections. After the 
step S2b only the object 03 and the actual specular re- 
flection SR are stilUdentified as potential specular re- 
flections PSR. 

[0027] A sub-step S2c determines the luminance gra- 
dient for each potential specular reflection PSR. This is 
accomplished, for example, using subtraction across 
the spatial extent of the potential specular reflection 
PSR. The luminance gradient is defined herein as the 
rate of change of luminance within each potential spec- 
ular reflection PSR. An actual specular reflection SR ex- 
hibits a strong luminance gradient as opposed to other 
bright white image regions such as ordinary white ob- 
jects. 

[0028] The sub-step S2d compares the brightness or 
luminance gradient of all potential specular reflections 
PSR to a select luminance gradient threshold. A step 



S2e discards from the set of all potential specular reflec- 
tions those not exhibiting a luminance gradient above 
the select luminance gradient threshold. In the present 
example, the object 03 in the image A is a white button 
or the like, and is not defined by a sufficiently strong lu- 
minance gradient to satisfy the luminance gradient 
threshold applied by the sub-step S2d and will thereafter 
be disregarded according to the sub-step S2e. 
[0029] The sub-step S2f identifies all remaining po- 
tential specular reflections PSR as specular reflections 
SR. In the present example, the sub-step S2f identifies 
only the actual specular reflection SR as a specular re- 
flection. If desired, an optional sub-step S2e can be pro- 
vided before the sub-step S2f. The sub-step S2e looks 
for red colored pixels adjacent the potential specular re- 
flection PSR. If none are identified, the potential spec- 
ular reflection is disregarded and not identified as a 
specular reflection SR. In such case, given that no red 
colored pixels are located adjacent the potential specu- 
lar reflection, further processing to locate red-eye asso- 
ciated with a specular reflection is unnecessary. 
[0030] It should be noted that false positives (i.e., 
identifying a region R as including a specular reflection 
SR when it does not, in fact include a specular reflection) 
are possible and even desirable. The downstream con- 
ventional red-eye detection and correction processing 
performed on the regions identified as including red-eye 
will identify those regions not actually including red-eye 
and will disregard same. On the other hand, by biasing 
the foregoing method toward identifying potential spec- 
ular reflections as specular reflections, the likelihood of 
missing an occurrence of red-eye is minimized. 
[0031] Those of ordinary skill in the art will recognize 
that the in cases where the region identification and sub- 
sequent red-eye detection algorithm delivers an uneven 
number of red-eyes, additional processing can be ap- 
plied, searching for rare cases of red-eye without ac- 
companying specular reflection. One those extensions 
would examine the image along essentially horizontal 
bands around identified red-eyes. 
[0032] The invention has been described with refer- 
ence to preferred embodiments. It should be noted that 
sub-steps 2b through 2f might be rearranged or in part 
omitted. Other modifications will occur to those of ordi- 
nary skill in the art to which the invention pertains upon 
reading and understanding of this specification. It is in- 
tended that the invention be construed as including all 
such modifications and alterations insofar as they come 
within the scope of the appended claims or the equiva- 
lents thereof. 



Claims 

1. A digital image processing method for identifying 
potential red-eye regions of a digital image, said 
method comprising: 



15 



20 



25 



30 



35 



40 



45 



50 



7 



EP 1 199 672 A2 



8 



receiving a digital image defined by a plurality 
of pixels; 

locating regions of interest within said digital 
image that include pixels that represent a spec- 
ular reflection; and, 

identifying each region of interest as a potential 
red-eye region. 

2. The digital image processing method as set forth in 
claim 1 , wherein said step of locating regions of in- 
terest comprises examining luminance-chromi- 
nance data for each pixel of said digital image. 

3. The digital image processing method as set forth in 
claim 2, further comprising, before said step of ex- 
amining luminance-chrominance data, converting 
each of said pixels of said digital image to a lumi- 
nance-chrominance color space. 

4. The digital image processing method as set forth in 
claim 2, wherein said step of locating regions of in- 
terest in said digital image comprises: 

identifying a group of adjacent pixels of said 
digital image as a potential specular reflection 
if said group of adjacent pixels comprises pixels 
having luminance-chrominance characteristics 
of a specular reflection; 
for each identified potential specular reflection, 
determining at least one geometric attribute of 
said potential specular reflection; 
comparing the at least one geometric attribute 
of said potential specular reflection to a desired 
geometric attribute; and, 
identifying said potential specular reflection as 
a specular reflection if said at least one geo- 
metric attribute conforms to said desired geo- 
metric attribute. 

5. The digital image processing method of claim 4, 
wherein the at least one geometric attribute of the 
potential specular reflection is at least one of a size 
and shape of said potential specular reflection. 

6. The digital image processing method as set forth in 
claim 4, further comprising, for each potential spec- 
ular reflection: 

determining a brightness gradient of said po- 
tential specular reflection; 
comparing said brightness gradient of said po- 
tential specular reflection to a select brightness 
gradient threshold; and, 
identifying said potential specular reflection as 
a specular reflection if said at least one geo- 
metric attribute conforms to said desired geo- 
metric attribute and said brightness gradient is 
greater than said brightness gradient threshold. 



7. A method for detecting red-eye in a digital image, 
said method comprising: 

receiving digital image data defining a digital 
5 image; 

identifying regions of interest in said digital im- 
age that include data that represent a specular 
reflection; 

examining the data in each of said regions of 
*o interest to detect red-eye. 

8. The red-eye detection method as set forth in claim 

7, wherein said step of identifying regions of interest 
comprises, for each region of an image: 

15 

identifying said region as including a specular 
reflection if said region includes a sub-region 
defined by at least one preset geometric at- 
tribute, a luminance above a select luminance 
20 threshold, and a luminance gradient above a 

select luminance gradient threshold. 

9. The red-eye detection method as set forth in claim 

8, wherein said at least one preset geometric at- 
25 tribute comprises at least one of a preset size and 

a preset shape. 

10. A method of detecting red-eye in a digital image, 
said method comprising: 



30 

receiving a digital image; 
identifying all regions of said digital image that 
include a specular reflection; 
for each region of an image including a specular 
35 reflection, determining if said region includes 

red-eye based upon a color of said region ad- 
jacent said specular reflection. 
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IDENTIFY ALL REGIONS INCLUDING 
A SPECULAR REFLECTION 
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PROCESS ONLY REGIONS INCLUDING 
A SPECULAR REFLECTION TO 
DETECT/CORRECT FOR RED-EYE 
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IDENTIFY ALL REGIONS INCLUDING 
A SPECULAR REFLECTION 
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IDENTIFY REGIONS INCLUDING POTENTIAL SPECULAR 
REFLECTIONS BASED UPON LUMINANCE/CHROMINANCE 
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DISREGARD REGIONS INCLUDING POTENTIAL SPECULAR 
REFLECTIONS ABOVE A SELECT SIZE THRESHOLD 



DETERMINE LUMINANCE GRADIENT IN REMAINING 
POTENTIAL SPECULAR REFLECTIONS 
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COMPARE LUMINANCE GRADIENT OF EACH POTENTIAL 
SPECULAR REFLECTION TO LUMINANCE GRADIENT THRESHOLD 



S2d 



DISREGARD REGIONS INCLUDING POTENTIAL SPECULAR 
REFLECTIONS HAVING LUMINANCE GRADIENT BELOW 
LUMINANCE GRADIENT THRESHOLD 



S2e 



CHECK COLOR OF REGION ADJACENT REMAINING POTENTIAL 
SPECULAR REFLECTIONS AND DISREGARD REGIONS WITHOUT 
RED ADJACENT POTENTIAL SPECULAR REFLECTIONS 
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IDENTIFY REMAINING POTENTIAL SPECULAR 
REFLECTIONS AS SPECULAR REFLECTIONS 
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